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Abstract: Autoregressive processes are intensively studied in statistics and other fields 
of applied stochastics. For many applications the overshoot and the threshold-time are of 
special interest. When the upward innovations are in the class of phasetype distributions 
^ we determine the joint distribution of this two quantities and apply this result to problems 

k£ of optimal stopping. Using a principle of continuous fit this leads to explicit solutions. 
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Autoregressive processes play an important role in many areas of applied probability and 
statistics. They can be seen as one of the building blocks for many models in time-series 
analysis and estimation and testing techniques are well developed. In this article we study 
the following setting: 

Let < A < 1, (Z n ) ne ^ be a sequence of independent and identically distributed random 
variables on a probability space (Q,A, P) and let (J r n ) n eN be the filtration generated by 



(Z n ) neN . Define the autoregressive process of order 1 (AR(l)-process) (A ra ) neN() by 

X n = AX n _i + Z n for all n G N 

X i.e. 

x n = \ n x + ^2 a z n _ k . 

k=0 

The random variables (Z n ) n£ fq are called the innovations of (X n ) ne ^ . Using the difference 
notation the identity X n = AX n _i + Z n can be written as 

AX n = -(1 - A)X n _xAn + AL n , 

where AX n = X n - X n _ u An = n - (n - 1) = 1 and AL n = ELi z k ~ Ej^J Z k = Z n . 
This shows that AR(l)-processes are the discrete-time analogon to (Levy-driven) Ornstein- 
Uhlenbeck processes. We just want to mention that many arguments in the following can 
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be carried over to Ornstein-Uhlenbeck processes as well. 

Autoregressive processes were studied in detail in the last decades. The joint distribution 
of the threshold-time 

T b = mf{n G N : X n > b} 

and the overshoot 

X Tb — b over a fixed level b 

was of special interest. If A = 1 the process (X n ) ng N is a random walk and many results 
about this distributions are well known. Most of them are based on techniques using the 
Wiener Hopf-factorization - see jH Chapter VII] for an overview. Unfortunately no anal- 
ogon to the Wiener- Hopf factorization is known for AR(l)-processes, so that other ideas 
are necessary. To get rid of well-studied cases we assume that A < 1 in the following. 

This first passage problem for AR(l)-processes was considered in different applications, 
such as signal detection and surveillance analysis, cf. [8]. In applications the distribution 
of this quantities are approximated using Monte-Carlo simulations or Markov chain ap- 
proximations, cf. e.g. [T6]. But e.g. for questions of optimization analytic solutions are 
necessary. 

Using martingale techniques exponential bounds for the expectation of r& can be found. 
Most of these results are based on martingales defined by using integrals of the form 

/ e uy -^u v ~ x du (1) 
Jo 

where <fi is the logarithm of the Laplace transform of the stationary distribution discussed 
in Section |3} For the integral to be well defined it is necessary that E(e uZl ) < 00 for all 
u G [0, 00) - cf. [11] and the references therein. 

On the other hand if one wants to obtain explicit results about the joint distribution of r& 
and the overshoot it is useful to assume Z\ to be exponentially distributed. In this case 
explicit results are given in 0, Section 3] by setting up and solving differential equations. 
Unfortunately in this case not all exponential moments of Z\ exist and the integral de- 
scribed above cannot be used. 

The contribution of this article is twofold: 

1. We find the joint distribution of 77, and the overshoot for a wide class of innovations: 
We assume that Z\ — S\ — T\, where S\ and T\ are independent, S\ has a phasetype 
distribution and T\ > is arbitrary. This generalizes the assumption of exponen- 
tially distributed innovations to a much wider class. In Section [2] we establish that 
Tfy and the overshoot are - conditioned on certain events - independent and we find 
the distribution of the overshoot. In Section [3] we use a series inspired by the integral 
([T]) to construct martingales with the objective of finding the distribution of Tb. 
This leads to explicit expressions for expectations of the form E x (p Tb g(X Tb )) for gen- 
eral functions g and p G (0, 1). 
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2. As an application we consider the (Markovian) problem of optimal stopping for 
(X n ) ngNo with discounted non-negative continuous gain function g, i.e. we study 
the optimization problem 

v(x) = sup E x (p T g(X T )) = supE(p T g(X T x + X T )), i6R, < p < 1, 
reT rer 

where T denotes the set of stopping times with respect to (J r n )neN ; to simplify 
notation here and in the following we set the payoff equal to on {r = oo}. Just 
very few results are known for this problem. In [TU] and [3] the innovations are 
assumed to be exponentially distributed and in [5] asymptotic results were given for 
g(x) = x. 

Following the approach described in [5] the problem can be reduced to determining 
an optimal threshold. This is summarized in Section |4j In a second step we use the 
joint distribution of T& and the overshoot to find the optimal threshold. To this end 
we use the principle of continuous fit, that is established in Section [6j An example 
is given in Section [7| 

2 Innovations of phasetype 

In this section we recall some basic properties of phasetype distributions and identify the 
connection to AR(l)-processes. In the first subsection we establish the terminology and 
state some well-known results, that are of interest for our purpose. All results can be found 
in [Tj discussed from the perspective of queueing theory. 

In the second subsection we concentrate on the threshold-time distribution for autoregres- 
sive processes when the positive part of the innovations is of phasetype. The key result for 
the next sections is that - conditioned to certain events - the threshold-time is independent 
of the overshoot and the overshoot is phasetype distributed as well. 

2.1 Definition and some properties 

Let m G N, E = {1, m}, A = m + 1 and E A = E U {A}. 

In this subsection we consider a Markov chain (Jt)t>o in continuous time with state space 
The states 1, ...,m are assumed to be transient and A is absorbing. Denote the 
generator of (J t )t>o by Q = {qij)i,j<zE A , i-e. 

qij(h) := P(J t +h = j\Jt = i) = qijh + o(h) for all i ^ j G E A and 
Qu(h) '■= P(Jt+h — i\ Jt — i) — 1 + Quh + °(h) f° r an i £ E A and h — > 0, t > 0. 

If we write Q(h) = (^))i,ieB A for all h > 0, then (Q(h))h>o is a semigroup and the 
general theory yields that 

Q(h) = e® h for all h>0. 
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Since A is assumed to be absorbing Q has the form 



Q 



0...0 



Q -Ql 



) 



for an m x m-matrix Q, where 1 denotes the column- vector with entries 1. 
We consider the survival time of (Jt)t>o, i>e. the random variable 



77 = inf{t >0:J t = A}. 



Let a = (a, 0) be an initial distribution of (Jt)t>o- Here and in the following a = (cti, a m ) 
is assumed to be a row-vector. 

Definition 2.1. is called a distribution of phasetype with parameters (Q,oe) and we 
write P? = PH(Q, a) for short. 

Let m = 1 and Q = {—(3) for a parameter > 0. In this case it is well-known that 77 
is exponentially distributed with parameter f3. This special case will be the key example 
we often think of. Furthermore let us mention that the class of phasetype distributions is 
stable under convolutions and mixtures. This shows that the important classes of Erlang- 
and hyperexponential distributions are of phasetype. 

Exponential distributions have a very special structure, but phasetype distributions are 
flexible: 

Proposition 2.1. The distributions of phasetype are dense in the space of all probability 
measures on (0, 00) with respect to convergence in distribution. 



The definition of phasetype distributions does not give rise to an obvious calculus 
with these distributions, but the theory of semigroups leads to simple formulas for the 
density and the Laplace-transform as the next lemma shows. All the formulas contain 
matrix exponentials. The explicit calculation of such exponentials can be complex in 
higher dimensions, but many algorithms are available for a numerical approximation. 

Proposition 2.2. (a) The eigenvalues of Q have negative real part. 

(b) The distribution function of PH(Q,a) is given by 



Proof. See [TJ III, Theorem 4.2]. 



□ 



H a (s) 



P a (v < s) = 1 - ae Qs l, s 



> 0. 



(c) The density is given by 



h a (s) = ae Qs q, s>0 



where q = —Ql. 
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(d) For all s G C with E & (e^ s)r) ) < oo it holds that 

H a (s) :=E & (e^) = a(-sI-Q)- 1 q, 

where I is the m x m-identity matrix. 
In particular H a is a rational function. 

Proof. See [TJ II, Corollary 4.9 and III, Theorem 4.1]. □ 

An essential property for the applicability of the exponential distribution in modeling 
and examples is the memoryless property, which even characterizes the exponential distri- 
bution. The next lemma can be seen as a generalization of this property to distributions 
of phasetype. 

Lemma 2.3. Let t > and write 

Hl(s) = P & (r] <s + t\rj>t) for all s>0. 

Then H l a is a distribution function of a phasetype distribution with parameters (Q,'K t ), 
where irj = Pa{Jt — i\v — t) for all % = 1, ...,m. 



Proof. By Proposition the random variable r\ has a continuous distribution. Therefore 
using the Markov-property of (Jt)t>o we obtain 

P&ij] <t + s\rj>i) = P & (rj <t + s\r]>t) 

= J2PeAv<s)nj = Y.<H et (s), 

where is the i-th unit vector. □ 

For the application to autoregressive processes we need the generalization of the previ- 
ous lemma to the case that the random variable is not necessarily positive. 

Lemma 2.4. Let S, T > be stochastically independent random variables, where S is 
PH(Q, a) -distributed. Furthermore let r > and Z = S — T . Then 

P&(r<Z<r + s) = J2 Xii^H^s), s > 0, 

ieE 

where A 4 (r) = / P&(J r+t = i)Pj(dt). 
Proof. Using Lemma |2.3| it holds that 

P&{r <Z<r + s) = J P & (r + t<S <r + s + t)PT(dt) 



J2 [H ei (s)P & (J t+r = i)pT(dt). 



□ 
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2.2 Phasetype distributions and overshoot of AR(l)-processes 

We again consider the situation of Section [T] In addition we assume the innovations to 
have the following structure: 

Z n = S n — T n for all n G N, 

where S n and T n are non-negative and independent and S n is PH(Q, a)-distributed. In 
this context we remark that each probability measure Q on K with Q({0}) = can be 
written as Q = Q + * Q- where Q + and Q- are probability measures with oo, 0)) = 

Q-((0, oo)) = and * denotes convolution (cf. [4, p. 383]). 

As a motivation we consider the case of exponentially distributed innovations. If Z n is 
exponentially distributed then it holds that for all p G (0, 1] and measurable g : R — > [0, oo) 

E x (pTg(X T )) = E x (p T )E(g(R + &)), (2) 

where r = is a threshold-time, x < b and R is exponentially distributed with the same 
parameter as the innovations (cf. [31 Theorem 3.1]). This fact is well known for random 
walks, cf. [U Chapter XII.]. The representation of the joint distribution of overshoot and 
r reduces to finding a explicit expression of the Laplace-transform of r. In this subsec- 
tion we prove that a generalization of this phenomenon holds in our more general situation. 

To this end we use an embedding of (X„) ng N into a stochastic process in continuous 
time as follows: 

For all n G N denote the Markov chain which generates the phasetype-distribution of S n 
by (Jt n) )t>o and write 

n 

J t = </j_v>«t q > where n t = max{n G No : V" Sk < t} for all £ > . 

k=l 

Hence the process (Jt)t>o is constructed by compounding the processes restricted to 
their lifetime. Obviously (Jt)t>o is a continuous time Markov chain with state space E, 
as one immediately checks, cf. [TJ III, Proposition 5.1]. Furthermore we define a process 
(Y t ) t >ohy 

Y t = \X nt — T nt+ i + t — ^2 Sk- 

k=l 

See Figure [T] for an illustration. It holds that 

X n = F(5 1+ ... +lSn) _ for all n G N, 

so that we can find (X n ) ngNo in (Y t ) t > . Now let f be the threshold-time of the process 
(Xt)t>o over the threshold b, i.e. 

f = inf{£ > : Y t > b}. 
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b 




By definition of (Y t ) t >o it holds that 

nt 

Y t = b & t = -XX nt + T nt+1 + J2Sk + bior all t > 0. (3) 

k=l 

For the following result we need the event that the associated Markov chain is in state 
i when (Y t ) t >o crosses b > 0, i.e. the event 

d = {J f = i} for % G E. 

For the following considerations we fix the threshold b > 0. 

In generalization of the result for exponential distributed innovations the following theorem 
states that - conditioned on Gi - the threshold-time and the overshoot are independent 
and the overshoot is phasetype distributed as well. 

Theorem 2.5. Let x <b, n <E N, y > and write 

t = r b = mf{n e N : X n > b}. 
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Then 

P X (X T -b<y,r = n) = J2 H ei (y)P x (r = n, G { ). 

ieE 

Proof. Using Lemma 2.4 and the identity ^ we obtain 



P X {X T -b<y,r = n)= E x {t {r > n} P x {X n >b,X n -b< y\jr n ^)) 

= E x (l {T > n} P x (b <X n <b + ylFn-t)) 
= E x (l {T > n} P x (b - XX n ^ <Z n <b + y- AX„_i|.F n _i)) 

= E x {l {T > n} H eM P x( J b-\X n - 1+ T n = *|^n-l)) 
ieE 

= J2 H ei(y)Px(r = n, J? = i). 

□ 

This immediately implies a generalization of (J2j) to the case of general phasetype dis- 
tributions: 

Corollary 2.6. It holds that 

E x (pTg(X T )) = J2E x (p T t Gt )E(g(b + R 1 )), 
ieE 

where R l is a Ph(Q,ei)- distributed random variable (under P). 

3 Explicit representations of the joint distribution of 
threshold time and overshoot 



Corollary 2.6 reduces the problem of finding expectations of the form E x (p T g(X T )) to find- 
ing = = E x (p T tci) for t = Tf, and b > x. The aim of this section is to construct 
martingales of the form (p nAT h(X nAr )) n( z N as a tool for the explicit representation of ^(x). 
To this end some definitions are necessary: 

We assume the setting of the previous section, i.e. we assume that the innovations can 
be written in the form 

Z n = S n — T n for all n G N, 

where S n and T n are non-negative and independent and S n is PH(Q, a)-distributed. 
Let exp(^) be the Laplace-transform of Z\, i.e. ip{u) = \ogE(e uZl ) for all u G C + := 
{z G C : > 0} with real part dt(u) so small that the expectation exists. Since 



E(e uZl ) = E(e uSl )E(e- uTl ) and T x > Proposition Q yields the existence of i/j(u) for all 
u with dt(u) smaller then the smallest eigenvalue of —Q. ip is analytic on this stripe and 
- because of independence - it holds that 

if)(u) = ipi(u) + if>2(u), 
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where exp(^i) denotes the Laplace-transform of Si and exp(ip 2 ) is the Laplace-transform 
of —T\. ip2 is analytic on C + and ipi can be analytically extended to C + \ Sp(—Q) by 



Proposition 2.2 Here Sp(-) denotes the spectrum, i.e. the set of all eigenvalues. Hence 
ip can be extended to C + \ Sp(—Q) as well and this extension is again denoted by if). 
Note that this extension can not be interpreted from a probabilistic point of view because 
E(e uZl ) does not exist for u G C + with too large real part. 

To guarantee the convergence of {X n ) n( z^ Q we assume a weak integrability condition - the 
well known Vervaat condition 



E{log{l + \Z 1 \)) <oo, (4) 

see [HI Theorem 2.1] for a characterization of such conditions in the theory of perpetuities. 
We do not go into details here, but just want to use the fact that (X n ) ngNo converges to a 
(finite) random variable 9 in distribution, that fulfills the stochastic fixed point equation 

pe _ p\e ^ pz 1 ^ 

where * denotes convolution. Since the AR(l)-process has the representation 

n-1 

X n = A n Xo + A Z n -fc 

k=0 

and convergence in distribution is equivalent to the pointwise convergence of the Laplace- 
transforms the Laplace-transform exp(0) of 9 fulfills 

oo 

= 5>(A*u) (5) 

k=0 

for all u G C + such that the Laplace-transform of S\ exists. 

The right hand side defines a holomorphic function on C + \ P that is also denoted by <f>, 
where we write P = UneN Sp(—X~ n Q). For the convergence of the series note that - as 
described above - it converges for all u G C + such that E(e uZl ) < oo. For all other u G C + 
the series also converges since there exists fco such that E(e x uZl ) < oo for all k>k$. 
Furthermore the identity 

<f>{u) = (j)(Xu) + if){u) (6) 

holds, whenever u, \u are in the domain of (p. To avoid problems concerning the applica- 
bility of (|6| we assume that 

Sp{X n Q) n Sp{Q) = for all n G N. (7) 

We would like to mention, that the function (f> was used and studied in [llj as well. 



The next two lemmas are helpful in the construction of the martingales. 
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Lemma 3.1. Let 5 G C + such that E(e SSl ) exists. Then for all x < b it holds that 

P E(e 5 ^ +z ^l {Xx+Zl y b} ) = a 5 e- XxQ q, 
where a s = pa(-5I - Q)-^ e (si+Q)b+M-Q) . 

Proof. In the following calculation we use the fact that all matrices commutate and that 
all eigenvalues of 51 + Q have negative real part. It holds 

rod 

E(e s ^ +z ^l {Xx+Sl > b} ) = e 5Xx J Q ^(e 5 ( 5 -*)l {Ax+ 5 1 - t > 6} )P T (dt) 

poo poo 

= e 5Xx / / e S{s - t) ae Qs qdsP T (dt) 

JO Jb+t-Xx 



e SXx r e~ 5t a f° e^ I+Q >dsqP 7 \dt) 

JO Jb+t-\x 
poo 

/ e- 6t a(5I + Q)- 1 e^ 5I+Q ^ b+t - x ^qP T (dt). 
Jo 



-e SXx 



We obtain 



roc 

pE(e s ^ +z ^t {Xx+Zl > b} ) = -pe SXx aj o e Qt P T (dt)(5I + Q)-y«+«><*-*») g 

= -pa [™ e- Qs p- T (ds)(5I + Q)- 1 e- QXx e( 5I+ ® b q 
Jo 



a s e- QXx q. 



□ 



We write Q 1 = — for short. For all 7 G C + fulfilling 

Sp(X n Q 1 ) n P = for all n G N (8) 

we define the function 

/ 7 : E -> C mxm ,x t-> ^ e ^A»Q 7 -^(A"Q 7 ) pn -i 

neN 

This series converges because e 3 ^' 1 ^-^"^) j s bounded in n. Note that the summand of 
this series is similar to the integrand in Q. 

Lemma 3.2. There exists 5 > such that for all x G 1R and 7 G C + I7I < 5 it holds 
that 

pEif^Xx + Zt)) = f 7 (x) - e XxQ ^ XQ -<\ 
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Proof. For all 7 G C + with |7| sufficiently small the expected value E(e® lXnZl ) exists for 
all n G N since Q has (finitely many) negative eigenvalues. This leads to 

E(f y (Xx + Z x )) = E(J2 e (^)A*Q T -<KA»Q T y-i) 

neN 

_ e xA"+ 1 Q 7 -0(A"Q 7 )+i/'(A' l Q 7 )^n-l 

<J| 1 e a;A"+ 1 Q 7 -(/»(A"+ 1 Q 7 )^n 
P n£N 



□ 



The next step is to find a family of equations characterizing 

$(x) = ($ 1 (a;),...,$ m (x)) = (E x (p T l Gl ),...,E x (p T l G J) 
using martingale techniques where t = t^, x < b. To this end we consider 



for all 5 G C + \ SP(-Q) and 7 fulfilling (M) where /3 7) a = c^e 9 ^^. For the special value 
7 = 1 we write ^5 = h lt s an d this function is well-defined by Q. 



Putting together the results of Lemma 3.1 and Lemma 3.2 we obtain the equation 



pE^h^X^) - h liS (x) = a s e- XxQ q - a s e~ XrxQ q for all x < b (9) 
for all 7, S G C + with sufficiently small modulus. 

Before stating the equations we need one more technical result. 

Lemma 3.3. Let R 1 be a PH(Q,ei)- distributed random variable and denote by = 
log(ej(— ■ / — Q)~ 1 q) the holomorphic extension of the logarithmized Laplace-transform of 
Ri. Here ei denotes the i-th unit vector. Let \^y\, \5\ be so small that E{hry^(b-\- R 1 )) exists. 
Then it holds that 

E(h 1:5 (b + if)) = e Sb a^(-5I - Q)- l q =: 

where 

a ^. = e . + ae Qb+^2(-Q)+<t>^Q-f) J2 e ^ n Qi-<t>{*> n Qi)+r^ n Qi) p n _ 

raeN 

Proof. Simple calculus similar to the previous ones yields the result. □ 
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Theorem 3.4. For all x < b and 6 G C + \ Sp(—Q) it holds that 

m 

^2r)s,i®i(x) = h s {x), 
i=i 

where rjg^ = 771,5,1 is given in the previous Lemma. 

Proof. Write h := h^s f° r <^7 C + with \5\, \ j\ so small that E(h(Zi)) < 00. 
The discrete version of Ito's formula yields 

n-l 



8=0 
n-l 

--h(X ) + P l+ \KX 1+1 ) - E x {h{X i+1 )\Xi)) =: M n 



i=0 



and (M n ) n6 N is a martingale. The optional sampling theorem applied to r = Tb yields 

/nAr-l \ 

h(x) = E x (M rAn ) = E x (p nAT h(X nAT )) - E x ^ P^pE.iKX^X,) - h(Xi))j 

/nAr-l \ 

= E x (p nAT h(X nAT )) - E x I p p>(a 5 e- xx < Q q-ase-^ x * Q q)j 

using equality ([9]). The dominated convergence theorem shows that 

h Sn {x) = E x {p T h Sn {X T )) - E x (j2p\a 5 e- xx * Q q - a 5 e~ x ^ Q q)^ ; 

note that the dominated convergence theorem is applicable to both summands since Q has 
negative eigenvalues and so e~ sQ is bounded in s for s with 5R(s) being bounded above. 
Corollary 2J3 leads to 

M*) = E E^ta^E^^Ri + b))-E x [Y J p l (ase- xx > Q q - a 5 e~ x ^ Q q) , 

i=l \i=0 / 

where Ri is PH(Q, e^)— distributed and the previous Lemma implies 

E{hy, s (Ri + b)) = e 5b a J:i (-5I - Q)~ x q. 
Since ZllJo 1 P l [oL^e~ XXiC ^q — ase~ XlXiQ q) is bounded both sides of the equation 

hs, 7 {x) = J2v-y,s,Mx) ~ E x [Y, p\a s e- xx * Q q - ase^^q) 

i=l Vi=0 / 
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are holomorphic in {7 G C + : Pfl Sp(—'yX n Q) = for all n G N} and the identity theorem 
for holomorphic functions yields that these extensions agree on their domains. Keeping 
(J7]) in mind we especially obtain for 7 = 1 

m 

i=i 

Furthermore both sides of the equations are again holomorphic functions in 6 on 
C + \ Sp(—Q). Another application of the identity theorem proves the assertion. □ 

The equation in the theorem above appears useful and flexible enough for the explicit 
solution as shown in the next subsections. 



3.1 The case of exponential positive innovations 

As described above the case of Exp(/i)-distributed positive innovations is of special interest. 
In this case we obtain the solution directly from the results above. Hence let m — 1, a = 1, 
Q = —ji and q = \i. It is not relevant which 5 we take; because the expressions simplify a 
bit we choose 5 = 0. Then we obtain 

rj Q 1 = 1 + e MlA-»b+<t>(^) e A>6-<jHA>)+V>i(A>)^n 

H 1 _|_ e V>2(M)-MM-<«A M ) e A n Mfc-</>(A™+V)-^2(A™M)pn 

= e ^ 2 (M)-Mb+0(A M ) e * n »b-<l>(\ n+1 IJ,)-M* n H) p n 

n£N 



and 



net 



for x < b. Theorem |3.4| yields 
Theorem 3.5. 

hn(r) V M p^^x-H^v) n n 

iIq\X) L n£ Ne P f a <b 

**V>) mi EngNoe A^(A"+V)-^(A«M) p n PrallxKb. 

In [3] the special case of positive exponential distributed innovations was treated by 
finding and solving ordinary differential equations for E x (p T ). For this case - i.e. T% = - 
we obtain 

„\ n tix-<p(\ n p.) n 
Z^neN e P 



E x (p T ) 



Z^neNn e P 
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To get more explicit results we need a simple expression for 0(A n /i). Using identity ^ we 
find such an expression as 



e 



?HA» = TT Vi(A"+V) = TT ft _ TT _ 

n^!(l-A fc ) _ (A, A)„._i 

" nr=l(l-A fc ) ~ 0e(A) ' 

where (a,q) n = E[fc=i(l ~ aq k ~ l ) denotes the g-Pochhammer-symbol and e (g) = (q, q) c 
denotes the Euler function. This leads to 



E x (p T ) 



^n&N (A,A) n _i 



V _T_ P \ n [ib 
Z^nGNo (A,A) n C 

and the numerator is given by 

y- P" c a>. = y- (P A T 

ngN (A, A)„_i fcgNo fc! ngN (A, A)„_i 

_^ (/i3;A) fc ^ (pA^"" 1 

fceNo ^ neN (A) A)n-1 

^ (px\) k 1 

P 



fceN 



fc! (pA fc , A) 



OC 



100 fceN 

Note that we used the g-binomial-theorem in the third step (see [H (1.3.15)] for a proof). 
An analogous calculation for the denominator yields 

y- P" c A>b = 1 y- (p, X) k (pb) k 
neNo ( A ' A )« fceNo fc! 

and we obtain 

Theorem 3.6. If Si is Exp(/i)- distributed and 7\ = z£ /ioWs i/tai 

£.3; p = p ■ y , L ; t . /or a// x < b. 

Z^fceNolP' A Jfc~ fcj - 

For the special case p = 1 this formula was obtained by an approach via differential 
equations based on the generator in (TUJ Theorem 3]. 
Noting that 

^^^•^^ 
SfceN„(P) A)fc— £p 
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by direct calculation we find 

j h E x {p Tb ) = E x (p^ME b (p^) - 1). 

This reproduces Theorem 3.3 in [3j. Note that in that article the stopping time f?, 
inf{n G N : X t > b} was considered. But this leads to analogous results since 

T b+ = % under P b 

and 

Tb = fb P x a-s. for all 1^6. 

3.2 The general case 



Theorem |3.4| gives a powerful tool for the explicit calculation of $ in many cases of interest 
as follows: 



By Lemma 3.3 we see that r/s^ is a rational function of 5 with poles in Sp(—Q) for all 
i = l,..,m. We assume for simplicity that all eigenvalues are pairwise different (for the 
general case see the remark at the end of the section). Then partial fraction decomposition 
yields the representation 

W = J2 ^ fOT SOIIle 

and since hs(x) is rational in 5 with the same poles we may write 



m cAx) 

hs(x) = y2— f for some ci(x),...,c m (x). 



Theorem 13.41 reads 



3=1 H ~ 5 3=1 N ~ 6 
and the uniqueness of the partial fraction decomposition yields 

m 

4=1 

i.e. 

A$(x) = c(x), 

where A = (0^)4^=1) c ( x ) — ( c j( x ))jLi- This leads to 
Theorem 3.7. If A is invertible, then is given by 

$(x) = A~ l c(x) for all x < b. 

Remark 3.8. Note that the assumption of distinct eigenvalues was made for simplicity 
only. When it is not fulfilled we can use the general partial fraction decomposition formula 
and obtain the analogous result. 
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4 Applications to optimal stopping 

To tackle the optimal stopping problem 

v(x) = sup E x (p T g(X T )) 
rer 

it is useful to reduce the (infinite dimensional) set of stopping times to a finite dimensional 
subclass. A often used class for this kind of problems is the class of threshold-times. This 
reduction can be carried out in two different ways: 

(a) We use elementary arguments to reduce the set of potential optimal stopping times to 
the subclass of threshold-times, i.e. to stopping times of the form 

r b = in£{t >0:X t >b} 

for some b G R. Then we find the optimal threshold. A summary of examples where 
is approach can be applied is given below. 

(b) We make the ansatz that the optimal stopping time is of threshold type, identify the 
optimal threshold and use a verification theorem to prove that this stopping time is 
indeed optimal. 

On ([a]): In [3J Section 2] the idea of an elementary reduction of optimal stopping prob- 
lems to threshold-problems was studied in detail. For arbitrary innovations this approach 
can be applied to power gain function, i.e. g{x) = x n ,n G N. This problem that is known 
as the Novikov-Shiryaev problem and was completely solved for random walks in [T2] and 
[T3] . Furthermore the approach applies to gain functions of call- type g(x) = (x — K) + . 
For the special case of nonnegative innovations a much wider class of gain functions can 
be handled such as exponential functions. 

On |b]): To use the second approach described above the following easy verification 
theorem is useful. 

Lemma 4.1. Let b* G R, write v*(x) = E x (p Tb * g(X Tbt )) and assume that 

(a) v*(x) > g(x) for all x < b* . 

(b) E(pv*(Xx + Z x )) < v*(x) for all x G R. 
Then v = v * and 7> is optimal. 

Proof. By the independence of (Z n ) n£ N property (6) implies that (p n t>*(A n ))„ e pj is a su- 
permartingale under each measure P x . Since it is positive the optional sampling theorem 
leads to 

v*(x) > sup E x (p T v*(X T )) > sup E x (p T g(X T )) for all x G R, 

where the second inequality holds by (a) since v*(x) = g(x) for all x > b. On the other 
hand v*(x) < v(x), i.e. v*(x) = v(x) and r b * is optimal. □ 



16 



5 On the explicit solution of the optimal stopping 
problem 

Now we are prepared to solve the optimal optimal stopping problem 

v(x) = sup E x (p T g(X T )), x e R. 

T 

Section [4] gives conditions for the optimality of threshold-times. In this cases we can 
simplify the problem to 

v{x) = supE x (p Tb g(X Tb )), xER. 

b 

Now take an arbitrary starting point x. Then we have to maximize the real function 

m 

* x : {x, 00) -> R, b ^ E x (p Tb g(X Tb )) = £ ^(x)E(g(b + if)), 

i=l 

where R l is Ph(ei, Q)-distributed, b > 0. The results of the previous section give rise to 
an explicit calculation of ^(x) and of ^> x . 

Hence we are faced with the well-studied maximization problem for real functions, that 
can - e.g. - be solved using the standard tools from differential calculus. 
If we have found a maximum point b* of ty x and ty x (b*) > g(x), then 

t* = inf{n E N : X n > b*} 

is an optimal stopping time when (X n ) ne ^ is started in x. 

A more elegant approach for finding the optimal threshold b* is the principle of continuous 
fit: 

6 The principle of continuous fit 

The principles of smooth and continuous fit play an important role in the study of many 
optimal stopping problems. The principle of smooth fit was already introduced in [H] and 
has been applied in a variety of problems, ranging from sequential analysis to mathematical 
finance. The principle of continuous pasting is more recent and was introduced in [JJ] as 
a variational principle to solve sequential testing and disorder problems for the Poisson 
process. For a discussion in the case of Levy processes and further references we refer to 
[2]. Another overview is given in [TSJ, Chapter IV. 9] and one may summarize, see the above 
reference, p. 49: 

"If X enters the interior of the stopping region S immediately after starting on dS, 
then the optimal stopping point x* is selected so that the value function v is smooth in x*. 
If X does not enter the interior of the stopping region immediately, then x* is selected so 
that v is continuous in x*." 
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Most applications of this principle involve processes in continuous time. In discrete time 
an immediate entrance is of course not possible, so that one can not expect the smooth- 
fit principle to hold. In this section we prove that the continuous-fit principle holds in 
our setting and illustrate how it can be used for an easy determination of the optimal 
threshold. 

We keep the notations and assumptions of the previous sections and - as before - we 
assume that the optimal stopping set is an interval of the form [6*,oo) and consider the 
optimal stopping time r b * = r = infjra G N : X n > b*}. 
Furthermore we assume that 

lim$f (b* -e) = lim$f +e (&*) for alH = 1, m. (10) 

Note that this condition is obviously fulfilled in the cases discussed above. If g is continuous 
under an appropriate integrability condition it furthermore holds that 

E(g(Ri + t + b*)) E(g(Ri + b*)) as e 0, i = l,...,m. (11) 



Proposition 6.1. Assume (10) and (11). Then it holds that 

lim v{b) =g{b*). 

Proof. Let e > 0. First note that v(b) > g(b) for all b < b* so that 

liminf v (b) > liminf g(b) = g(b*). 

b/*b* b/*b* 



Furthermore using Corollary 2.6 



v(b* - e) - g(b*) = E b *_ e (pTg(X T )) - v(b*) 

< E b *_ t (pTg(X T ))-E b *(p^g(X Tb *J) 



Y, (®f{b* ~ e)E(g(Ri + &*)) - $f ^{b*)E{g{R i + e + &*))) 



as e \ 0. This proves limsup b/ >, fe * v(b) > g(b*). □ 

Figure [2] illustrates how the continuous fit principle can be used: We consider the 
candidate solutions 

'* x (6) , x < b 
g(x) , x > b 



v(b, x) 



and solve the equation ^ b _(b) = g{b), where \F is defined as in the previous section. If 
the equation has a unique solution we can conclude that this solution must be the optimal 
threshold as illustrated in the following Section. 
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0,5 1 1,5 

x 

| v(1.5,x) v(0.2,x) v(x)| 

Figure 2: Some candidate solutions for different thresholds in the case g(x) = x 



7 Example 

We consider the gain function g(x) = x and Exp(/i)-distributed innovations; in this setting 
we always assume (X n ) ng N to have values in [0, oo). The discussion in Section [^guarantees 
that the optimal stopping time is of threshold-type. The optimal threshold can be found 
by the continuous fit principle described in the previous section: 
The problem is solved if we find a unique b* G [0, oo) that solves the equation 

6 = * s _(6) = »_(6)(6 + i) = ^^^gf (6 + I), 



where we used Theorem 3.6 in the last step. This equation is equivalent to 
k=o /i - fc=o fc=o A* 



i.e. /(6) = 0, 



where 
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Note that /(0) = £ > and 



00 n k n\ k+l 

f(b) = - £(p, A Wf[ C 1 - X* + ^ < for a11 b E [°« °°)' 

Since /(fe) < - — (1 — p)(l — p\)b we furthermore obtain f(b) — > —00 for b — > 00. Hence 
there exists a unique solution b* of the transcendental equation f(b) =0. 
The optimal stopping time is 

t* = inf{n G N : X n > b*} 
and the value function is given by 



x , x > b*. 



In Figure |2]w is plotted for the parameters /x = l,p = A = l/2. 
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